Weighted N-gram model for evaluating Machine Translation output

I present the results of an experiment on extending an automatic method of Machine Translation evaluation (BLEU) with weights for the statistical significance of lexical items. I show that this extension gives additional information about evaluated texts; in particular it allows us to measure translation Adequacy, which, for statistical MT systems, is often overestimated by the baseline BLEU method. The proposed model also improves the stability of evaluation scores with a single human reference translation, which increases the usability of the proposed method for practical purposes. The model suggests linguistic a interpretation which develops deeper understanding of human intuitions about translation Adequacy and Fluency.